Components in Dynamic Video Content

The fast expansion of Internet and DVB channels has brought a fast increase of video footage which needs to be indexed for efficient and easy retrieval. This task has been historically done by documentalists who tag manually each video with a few keywords, unfortunately such work is time consuming and hence very expensive. In the last decade much effort has been put into building processes which automatically assign content-based labels to video documents, a proof of this is the existence of the TRECVid Video Retrieval Evaluation [1] workshops since 2003. Structural similarity metrics for still images has been largely studied lately, the goal of these is discovering underlying structure of an image that is impervious to rotations, translations, resizing and other transformation. This way images can be easily compared without being affected by their different scales and any kind of intermediate processing that they have experienced. The question tackled in this work is whether something analogous can be made for video, so similar videos can be detected independently of their size, frame rate and image content. Video indexing for retrieval is an old concept, first approaches date from the first half of the nineties. Back in 1994, Smoliar et al. [2] already stated the necessity for video software to identify and represent video content for indexing and retrieval. In parallel to video indexing, video classification has also brought much attention, while retrieval focus on finding videos in a database that match a given query, classification puts all the input videos into predefined categories, which are labeled. There are many ways to address these issues, mainly there have been three fields of research. One is text-based approach, which is based on identifying text objects and processing them with optical character recognition or extracting text from closed-captions, like did Wei Qi et al. [3] to automatically categorize news stories. Another approach is using audio features, processing the data can be made in time domain (energy, zero crossings): E. Wold et al. [4], Z. Liu et al. [5]; or in the frequency domain (bandwidth, frequency centroid, pitch): U. Srinivasan et al. [6]; a strong point in this case is the maturity of audio processing techniques. Finally, the third field of research is using visual information, which attracts a great deal of interest as most of the information processed by humans comes from their vision and because despite the efforts, the Human Visual Systems …